Skip to content

fix(tests): use .invalid TLD for unresolvable DNS test#12416

Merged
Dreamsorcerer merged 3 commits intoaio-libs:masterfrom
Bojun-Vvibe:fix/aio-libs-aiohttp-11293
May 4, 2026
Merged

fix(tests): use .invalid TLD for unresolvable DNS test#12416
Dreamsorcerer merged 3 commits intoaio-libs:masterfrom
Bojun-Vvibe:fix/aio-libs-aiohttp-11293

Conversation

@Bojun-Vvibe
Copy link
Copy Markdown
Contributor

@Bojun-Vvibe Bojun-Vvibe commented Apr 23, 2026

Closes #11293

What do these changes do?

Switch the unresolvable-DNS test hostname from wrong-dns-name.com to wrong-dns-name.invalid, using the reserved .invalid TLD (RFC 2606), so the test cannot be flaked by NXDOMAIN hijacking, parking pages, or cached records on networks that opportunistically resolve .com typos.

Also added a "never executed" message to the inner assert False to match the sibling test and aid debugging when something does enter the body.

Are there changes in behavior for the user?

No — test-only change. No public API or runtime behavior is affected.

Is it a substantial burden for the maintainers to support this?

No — single hostname swap in one test, no new dependencies, and .invalid is guaranteed unresolvable by RFC 2606, so there is no new infrastructure to maintain.

Related issue number

Fixes #11293

Root cause

test_aiohttp_request_ctx_manager_not_found relied on http://wrong-dns-name.com failing DNS resolution, but on some networks (notably macOS / certain ISPs) that domain can intermittently resolve. When it resolves, the request proceeds past the async with, the inner assert False trips, and the outer pytest.raises(ClientConnectionError) fails.

Regression test

tests/test_client_functional.py::test_aiohttp_request_ctx_manager_not_found — same test, now deterministic; it asserts that entering an aiohttp.request context manager for an unresolvable host raises ClientConnectionError and never enters the body.

Verification

Manually verified via socket.getaddrinfo('wrong-dns-name.invalid', 80) raises gaierror, confirming .invalid is unresolvable and the test will reliably hit the ClientConnectionError path. Local pytest env was missing some required plugins (textual), so the in-tree run is left to CI.

Checklist

  • I think the code is well written
  • Unit tests for the changes exist (the existing test, now deterministic)
  • Documentation reflects the changes (N/A — test-only)
  • If you provide code modification, please add yourself to CONTRIBUTORS.txt
  • Add a new news fragment into the CHANGES/ folder (CHANGES/11293.misc.rst is included in this PR)

test_aiohttp_request_ctx_manager_not_found was flaky on networks where
'wrong-dns-name.com' could occasionally resolve (e.g. ISP NXDOMAIN
hijacking or transient parking entries), causing the request to
proceed and trip the inner assertion. Switch to the reserved .invalid
TLD (RFC 2606), which is guaranteed never to resolve.
@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label Apr 23, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 23, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.92%. Comparing base (365f717) to head (ac5b6d1).
⚠️ Report is 17 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #12416      +/-   ##
==========================================
- Coverage   98.92%   98.92%   -0.01%     
==========================================
  Files         134      134              
  Lines       46751    46750       -1     
  Branches     2430     2429       -1     
==========================================
- Hits        46249    46248       -1     
  Misses        373      373              
  Partials      129      129              
Flag Coverage Δ
CI-GHA 98.98% <100.00%> (-0.01%) ⬇️
OS-Linux 98.72% <100.00%> (-0.01%) ⬇️
OS-Windows 96.98% <100.00%> (-0.01%) ⬇️
OS-macOS 97.88% <100.00%> (-0.01%) ⬇️
Py-3.10.11 97.39% <100.00%> (-0.01%) ⬇️
Py-3.10.20 97.86% <100.00%> (-0.01%) ⬇️
Py-3.11.15 98.11% <100.00%> (-0.01%) ⬇️
Py-3.11.9 97.65% <100.00%> (-0.01%) ⬇️
Py-3.12.10 97.73% <100.00%> (-0.01%) ⬇️
Py-3.12.13 98.20% <100.00%> (-0.01%) ⬇️
Py-3.13.13 98.44% <100.00%> (-0.01%) ⬇️
Py-3.14.4 98.51% <100.00%> (+<0.01%) ⬆️
Py-3.14.4t 97.51% <100.00%> (-0.01%) ⬇️
Py-pypy3.11.15-7.3.21 97.35% <100.00%> (+<0.01%) ⬆️
VM-macos 97.88% <100.00%> (-0.01%) ⬇️
VM-ubuntu 98.72% <100.00%> (-0.01%) ⬇️
VM-windows 96.98% <100.00%> (-0.01%) ⬇️
cython-coverage 38.08% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Apr 23, 2026

Merging this PR will not alter performance

✅ 67 untouched benchmarks
⏩ 4 skipped benchmarks1


Comparing Bojun-Vvibe:fix/aio-libs-aiohttp-11293 (ac5b6d1) with master (b6abd1b)2

Open in CodSpeed

Footnotes

  1. 4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on master (4cd1b36) during the generation of this report, so b6abd1b was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@Bojun-Vvibe Bojun-Vvibe marked this pull request as ready for review April 24, 2026 15:24
Comment thread CHANGES/11293.misc.rst Outdated
Comment thread CHANGES/11293.misc.rst Outdated
Comment thread tests/test_client_functional.py Outdated
Comment thread tests/test_client_functional.py Outdated
Copy link
Copy Markdown
Member

@webknjaz webknjaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fill out the PR template.

- CHANGES/11293.misc.rst: drop copilot credit, switch RFC link to :rfc: role
- tests/test_client_functional.py: revert verbose comment & assertion message

Per webknjaz review on aio-libs#12416

Signed-off-by: Bojun Chai <bojunchai@microsoft.com>
@Bojun-Vvibe
Copy link
Copy Markdown
Contributor Author

Thanks for the review @webknjaz — pushed e127668 addressing all four notes:

  • Dropped the copilot credit and switched the RFC reference to the :rfc: role in CHANGES/11293.misc.rst.
  • Reverted the verbose explanatory comment and the "never executed" assertion message in tests/test_client_functional.py so the diff is now minimal — only the hostname changes from wrong-dns-name.com to wrong-dns-name.invalid.

Let me know if you'd like the PR description filled out against the template as well.

@Bojun-Vvibe
Copy link
Copy Markdown
Contributor Author

@webknjaz thanks — PR description filled out against the template. The change itself is just the hostname swap to .invalid (RFC 2606) plus the matching news fragment in CHANGES/11293.misc.rst. Happy to also add myself to CONTRIBUTORS.txt in the same PR if that's preferred.

@Dreamsorcerer Dreamsorcerer added backport-3.14 Trigger automatic backporting to the 3.14 release branch by Patchback robot bot:chronographer:skip This PR does not need to include a change note and removed bot:chronographer:provided There is a change note present in this PR labels May 4, 2026
@Dreamsorcerer Dreamsorcerer merged commit 5503528 into aio-libs:master May 4, 2026
49 checks passed
@patchback
Copy link
Copy Markdown
Contributor

patchback Bot commented May 4, 2026

Backport to 3.14: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 5503528 on top of patchback/backports/3.14/5503528282ee13a2fcdad13b641e38c3a18eaf81/pr-12416

Backporting merged PR #12416 into master

  1. Ensure you have a local repo clone of your fork. Unless you cloned it
    from the upstream, this would be your origin remote.
  2. Make sure you have an upstream repo added as a remote too. In these
    instructions you'll refer to it by the name upstream. If you don't
    have it, here's how you can add it:
    $ git remote add upstream https://github.com/aio-libs/aiohttp.git
  3. Ensure you have the latest copy of upstream and prepare a branch
    that will hold the backported code:
    $ git fetch upstream
    $ git checkout -b patchback/backports/3.14/5503528282ee13a2fcdad13b641e38c3a18eaf81/pr-12416 upstream/3.14
  4. Now, cherry-pick PR fix(tests): use .invalid TLD for unresolvable DNS test #12416 contents into that branch:
    $ git cherry-pick -x 5503528282ee13a2fcdad13b641e38c3a18eaf81
    If it'll yell at you with something like fatal: Commit 5503528282ee13a2fcdad13b641e38c3a18eaf81 is a merge but no -m option was given., add -m 1 as follows instead:
    $ git cherry-pick -m1 -x 5503528282ee13a2fcdad13b641e38c3a18eaf81
  5. At this point, you'll probably encounter some merge conflicts. You must
    resolve them in to preserve the patch from PR fix(tests): use .invalid TLD for unresolvable DNS test #12416 as close to the
    original as possible.
  6. Push this branch to your fork on GitHub:
    $ git push origin patchback/backports/3.14/5503528282ee13a2fcdad13b641e38c3a18eaf81/pr-12416
  7. Create a PR, ensure that the CI is green. If it's not — update it so that
    the tests and any other checks pass. This is it!
    Now relax and wait for the maintainers to process your pull request
    when they have some cycles to do reviews. Don't worry — they'll tell you if
    any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-slop backport-3.14 Trigger automatic backporting to the 3.14 release branch by Patchback robot bot:chronographer:skip This PR does not need to include a change note

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test_aiohttp_request_ctx_manager_not_found fails intermittently

4 participants